Multi-modal program inference: a marriage of pre-trained language models and component-based synthesis

Multi-modal program synthesis refers to the task of synthesizing programs (code) from their specification given in different forms, such as a combination natural language and examples. Examples provide precise but incomplete specification, provides an ambiguous more "complete" description. Machine-learned pre-trained models (PTMs) are adept at handling language, struggle with generating syntactically semantically code. Program techniques can generate correct code, often even specifications, examples, they unable work ambiguity languages. We present approach that combines PTMs component-based (CBS): used candidates description task, which then guide CBS procedure find matches examples-based specification. use our instantiate multi-modal systems for two programming domains: domain regular expressions CSS selectors. Our evaluation demonstrates effectiveness domain-agnostic comparison state-of-the-art specialized system, generality providing examples domains.